Cue duration and trial spacing effects in contingency assessment in the streaming procedure with humans.

According to the cycle/trial (C/T) rule, the rate of associative learning is a function of the ratio between the overall rate of U.S. presentation (C) and its rate in the presence of the conditioned stimulus (CS; [T]). This rule is well supported in studies with nonhumans. The present study was conducted to test whether it also applies to human contingency learning. In Experiment 1, participants were exposed to rapid streams of trials. Sensitivity to the cue-outcome contingency varied with both intertrial interval (ITI, which captures C) and cue duration, but the C/T rule was not respected, notably because the effect of ITI was much larger than the effect of cue duration. Experiment 2 showed that mere suppression of verbal strategies did not alter the magnitude of the ITI effect. Experiment 3 replicated Experiment 1 but with cue duration and ITI varied between 1,000 and 3,000 ms instead of between 100 and 1,000 ms. Performance was insensitive to both cue duration and ITI. This was not the consequence of Experiment 3 only varying the cue duration to ITI ratio by a factor of 3; in Experiment 4 where the cue duration was 100 ms, a 300-ms ITI was sufficient to observe an ITI effect. The lack of an ITI effect with a 1,000-ms cue and an ITI varying between 1,000 and 3,000 ms was replicated in Experiment 5. These results are discussed in light of how processes underlying associative learning might break down when events occur very rapidly. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

An analysis of reinstatement after extinction of a conditioned taste aversion.

Taste aversion learning has sometimes been considered a specialized form of learning. In several other conditioning preparations, after a conditioned stimulus (CS) is conditioned and extinguished, reexposure to the unconditioned stimulus (US) by itself can reinstate the extinguished conditioned response. Reinstatement has been widely studied in fear and appetitive Pavlovian conditioning, as well as operant conditioning, but its status in taste aversion learning is more controversial. Six taste-aversion experiments with rats therefore sought to discover conditions that might encourage it there. The results often yielded little to no evidence of reinstatement, and we also found no evidence of concurrent recovery, a related phenomenon in which responding to a CS that has been conditioned and extinguished is restored if a second CS is separately conditioned. However, a key result was that reinstatement occurred when the conditioning procedure involved multiple closely spaced conditioning trials that could have allowed the animal to learn that a US presentation signaled or set the occasion for another trial with a US. Such a mechanism is precluded in many taste aversion experiments because they often use very few conditioning trials. Overall, the results suggest that taste aversion learning is experimentally unique, though not necessarily biologically or evolutionarily unique. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Developmental trajectory of social reward motivation from early adolescence into adulthood in female and male Long-Evans rats.

Most studies of adolescent and adult behavior involved one age group of each, whereas the dynamic changes in brain development suggest that there may be behavioral flux in adolescence. In two studies, we investigated developmental changes in social reward motivation in female and male Long-Evans rats from prepuberty to early adulthood in a social operant conditioning task. Given the earlier onset of puberty in females than in males, we predicted the course of social reward development would differ between the sexes. Overall, the pattern of results from both studies suggests that the trajectory of social motivation across adolescence is characterized by upward and downward shifts that do not depend on the sex of the rats. During training, in both studies, the mean number of social gate openings and percentage of social gate openings was higher at P30 (prepubertal, early adolescence) and P50 (late adolescence) than at P40 (mid adolescence) and P70 (adulthood) irrespective of sex. Nevertheless, the specific age comparisons that were significant depended on the study. In both studies, P30 rats had greater levels of social motivation than did adults in accessing a social reward when increased effort was required (progressive ratio tests). In an extinction test, only P30 and P50 rats continued to show more nose-pokes at the previously social gate than at the nonsocial gate, suggesting resistance to extinction. The results highlight the importance of characterizing behavior at several timepoints in adolescence to understand the neural mechanisms, many of which show similar discontinuities as they develop across adolescence.

Intolerance of uncertainty affects the behavioral and neural mechanisms of higher generalization.

Intolerance of uncertainty (IU) is associated with several anxiety disorders. In this study, we employed rewards and losses as unconditioned positive and negative stimuli, respectively, to explore the effects of an individual's IU level on positive and negative generalizations using magnetic resonance imaging technology. Following instrumental learning, 48 participants (24 high IU; 24 low IU) were invited to complete positive and negative generalization tasks; their behavioral responses and neural activities were recorded by functional magnetic resonance imaging. The behavior results demonstrated that participants with high IUs exhibited higher generalizations to both positive and negative cues as compared with participants having low IUs. Neuroimaging results demonstrated that they exhibited higher activation levels in the right anterior insula and the default mode network (i.e. precuneus and posterior cingulate gyrus), as well as related reward circuits (i.e. caudate and right putamen). Therefore, higher generalization scores and the related abnormal brain activation may be key markers of IU as a vulnerability factor for anxiety disorders.

Goal-directed and flexible modulation of syllable sequence within birdsong.

Songs constitute a complex system of vocal signals for inter-individual communication in songbirds. Here, we elucidate the flexibility which songbirds exhibit in the organizing and sequencing of syllables within their songs. Utilizing a newly devised song decoder for quasi-real-time annotation, we execute an operant conditioning paradigm, with rewards contingent upon specific syllable syntax. Our analysis reveals that birds possess the capacity to modify the contents of their songs, adjust the repetition length of particular syllables and employing specific motifs. Notably, birds altered their syllable sequence in a goal-directed manner to obtain rewards. We demonstrate that such modulation occurs within a distinct song segment, with adjustments made within 10 minutes after cue presentation. Additionally, we identify the involvement of the parietal-basal ganglia pathway in orchestrating these flexible modulations of syllable sequences. Our findings unveil an unappreciated aspect of songbird communication, drawing parallels with human speech.

How do animals weigh conflicting information about reward sources over time? Comparing dynamic averaging models.

Optimal foraging theory suggests that animals make decisions which maximize their food intake per unit time when foraging, but the mechanisms animals use to track the value of behavioral alternatives and choose between them remain unclear. Several models for how animals integrate past experience have been suggested. However, these models make differential predictions for the occurrence of spontaneous recovery of choice: a behavioral phenomenon in which a hiatus from the experimental environment results in animals reverting to a behavioral allocation consistent with a reward distribution from the more distant past, rather than one consistent with their most recently experienced distribution. To explore this phenomenon and compare these models, three free-operant experiments with rats were conducted using a serial reversal design. In Phase 1, two responses (A and B) were baited with pellets on concurrent variable interval schedules, favoring option A. In Phase 2, lever baiting was reversed to favor option B. Rats then entered a delay period, where they were maintained at weight in their home cages and no experimental sessions took place. Following this delay, preference was assessed using initial responding in test sessions where levers were presented, but not baited. Models were compared in performance, including an exponentially weighted moving average, the Temporal Weighting Rule, and variants of these models. While the data provided strong evidence of spontaneous recovery of choice, the form and extent of recovery was inconsistent with the models under investigation. Potential interpretations are discussed in relation to both the decision rule and valuation functions employed.

Simulated operant reflex conditioning environment reveals effects of feedback parameters.

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, biological variability, and reward threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), biological variability (low, high), and reward threshold (easy, moderate, difficult) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

A retrospective analysis of the relation between resurgence and renewal of behavior targeted for reduction.

Functional communication training (FCT) is an evidence-based treatment for behavior targeted for reduction that often combines extinction for target responses and arranges functionally equivalent reinforcement for alternative behavior. Long-term effectiveness of FCT can become compromised when transitioning from clinic to nonclinic contexts or thinning reinforcement schedules for appropriate behavior. Such increases in targeted behavior have been conceptualized as renewal and resurgence, respectively. The relation between resurgence and renewal has yet to be reported. Therefore, the present report retrospectively analyzed the relation between renewal and resurgence in data collected when implementing FCT with children diagnosed with developmental disabilities. We found no relation when evaluating all 34 individuals assessed for resurgence and renewal or a subset of individuals exhibiting both resurgence and renewal. These findings suggest that one form of relapse may not be predictive of another form of relapse.

Adolescents flexibly adapt action selection based on controllability inferences.

From early in life, we encounter both controllable environments, in which our actions can causally influence the reward outcomes we experience, and uncontrollable environments, in which they cannot. Environmental controllability is theoretically proposed to organize our behavior. In controllable contexts, we can learn to proactively select instrumental actions that bring about desired outcomes. In uncontrollable environments, Pavlovian learning enables hard-wired, reflexive reactions to anticipated, motivationally salient events, providing "default" behavioral responses. Previous studies characterizing the balance between Pavlovian and instrumental learning systems across development have yielded divergent findings, with some studies observing heightened expression of Pavlovian learning during adolescence and others observing a reduced influence of Pavlovian learning during this developmental stage. In this study, we aimed to investigate whether a theoretical model of controllability-dependent arbitration between learning systems might explain these seemingly divergent findings in the developmental literature, with the specific hypothesis that adolescents' action selection might be particularly sensitive to environmental controllability. To test this hypothesis, 90 participants, aged 8-27, performed a probabilistic-learning task that enables estimation of Pavlovian influence on instrumental learning, across both controllable and uncontrollable conditions. We fit participants' data with a reinforcement-learning model in which controllability inferences adaptively modulate the dominance of Pavlovian versus instrumental control. Relative to children and adults, adolescents exhibited greater flexibility in calibrating the expression of Pavlovian bias to the degree of environmental controllability. These findings suggest that sensitivity to environmental reward statistics that organize motivated behavior may be heightened during adolescence.

Instrumental successive negative contrast in rats: Trial distribution, reward magnitude, and prefrontal cortex activation.

Successive negative contrast (SNC) has been used to study reward relativity, reward loss, and frustration for decades. In instrumental SNC (iSNC), the anticipatory performance of animals downshifted from a large reward to a small reward is compared to that of animals always reinforced with the small reward. iSNC involves a transient deterioration of anticipatory behavior in downshifted animals compared to unshifted controls. There is scattered information on the optimal parameters to produce this effect and even less information about its neural basis. Five experiments with rats trained in a runway to collect food pellets explored the effects of trial distribution (massed or spaced), amount of preshift training, reward disparity, and reward magnitude on the development of an iSNC effect. Start, run, and goal latencies were measured. Using spaced trials (one trial per day), evidence of the iSNC effect was observed with 24 preshift trials and a 32-to-4 pellet disparity. With massed trials (4 trials per session separated by 30-s intertrial intervals), evidence of iSNC was found with 12 preshift sessions (a total of 48 trials) and a 16-to-2 pellet disparity. The massed-training procedure was then used to assess neural activity in three prefrontal cortex areas using c-Fos expression in animals perfused after the first downshift session. There was evidence of increased activation in the anterior cingulate cortex and a trend toward increased activation in the infralimbic and prelimbic cortices. These procedures open a venue for studying the neural basis of the instrumental behavior of animals that experience reward loss.

Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex.

Deciding on a course of action requires both an accurate estimation of option values and the right amount of effort invested in deliberation to reach sufficient confidence in the final choice. In a previous study, we have provided evidence, across a series of judgment and choice tasks, for a dissociation between the ventromedial prefrontal cortex (vmPFC), which would represent option values, and the dorsomedial prefrontal cortex (dmPFC), which would represent the duration of deliberation. Here, we first replicate this dissociation and extend it to the case of an instrumental learning task, in which 24 human volunteers (13 women) choose between options associated with probabilistic gains and losses. According to fMRI data recorded during decision-making, vmPFC activity reflects the sum of option values generated by a reinforcement learning model and dmPFC activity the deliberation time. To further generalize the role of the dmPFC in mobilizing effort, we then analyze fMRI data recorded in the same participants while they prepare to perform motor and cognitive tasks (squeezing a handgrip or making numerical comparisons) to maximize gains or minimize losses. In both cases, dmPFC activity is associated with the output of an effort regulation model, and not with response time. Taken together, these results strengthen a general theory of behavioral control that implicates the vmPFC in the estimation of option values and the dmPFC in the energization of relevant motor and cognitive processes.

Pavlovian-to-instrumental transfer in individuals with chronic pain.

Avoidance of pain has been argued to be key factor leading pain events to chronic disability. In this respect, research has focused on investigating the working mechanisms of avoidance's acquisition. Avoidance of painful stimuli has been traditionally studied using a combination of Pavlovian and Instrumental procedures. However, such approach seems to go against real-life scenarios where avoidance is commonly acquired more readily. Using a novel pain avoidance paradigm, we tested whether pain avoidance can be installed in absence of associations between a cue and pain omission, and whether such avoidance differs between pain patients and healthy controls. Participants first learned to avoid painful stimuli by pressing a grip bar. Then, they passively encountered pairings of one geometrical shape with pain and of another geometrical shape without pain. Lastly, participants encountered the geometrical shapes while being able to use the grip bar. Results showed that participants pressed the bar more vigorously when encountering the previously pain-related shape compared to the pain-unrelated shape. This effect did not seem to differ between pain patients and healthy control. Our study could inspire a new way in measuring avoidance in pain, possibly paving the way to better understanding how avoidance is installed in chronic pain.

[Learning and Neural Activity: Beyond Localization].

Learning is classified into two types: "classical conditioning," which modifies simple reflexes, and "operant conditioning," which modifies complex voluntary behaviors. The neural circuits underlying these two types differ significantly. During the learning process of operant conditioning tasks, various changes in firing rate and firing synchrony of neurons can be observed across multiple brain regions. Additionally, neuronal firing rate and synchrony in several brain regions can be voluntarily controlled through operant conditioning. Consequently, it is evident that neurons in widespread brain regions have the potential for plastic changes to facilitate learning. It may be suggested that the learning of complex voluntary behaviors is underlined by widespread dynamic changes in neural activity and is not restricted to only a few brain regions.

Resurgence Following Higher or Lower Quality Alternative Reinforcement.

Resurgence is a temporary increase in a previously suppressed target behavior following a worsening in reinforcement conditions. Previous studies have examined how higher rates or magnitudes of alternative reinforcement affect suppression of the target behavior and subsequent resurgence. However, there has been no investigation of the effects of higher versus lower qualities of alternative reinforcement on resurgence. Using a three-phase resurgence preparation with rats, the present experiments examined the effects of an alternative reinforcer that was of higher (Experiment 1) or lower (Experiment 2) quality than the reinforcer that had previously maintained the target behavior. The results of both experiments showed greater reductions in target behavior with a higher quality alternative reinforcer and larger increases in target responding when a higher quality alternative reinforcer was removed. Along with prior findings with higher rates and magnitudes of alternative reinforcement, these findings suggest that variations in reinforcer dimensions that increase the efficacy of alternative reinforcement also tend to increase resurgence when alternative reinforcement is removed. The results are discussed in terms of the resurgence as choice in context model and in terms of potential clinical implications.

Striatal dopamine release tracks the relationship between actions and their consequences.

The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.

Different Effects of Peer Sex on Operant Responding for Social Interaction and Striatal Dopamine Activity.

When rats are given discrete choices between social interactions with a peer and opioid or psychostimulant drugs, they choose social interaction, even after extensive drug self-administration experience. Studies show that like drug and nondrug food reinforcers, social interaction is an operant reinforcer and induces dopamine release. However, these studies were conducted with same-sex peers. We examined if peer sex influences operant social interaction and the role of estrous cycle and striatal dopamine in same- versus opposite-sex social interaction. We trained male and female rats (n = 13 responders/12 peers) to lever-press (fixed-ratio 1 [FR1] schedule) for 15 s access to a same- or opposite-sex peer for 16 d (8 d/sex) while tracking females' estrous cycle. Next, we transfected GRAB-DA2m and implanted optic fibers into nucleus accumbens (NAc) core and dorsomedial striatum (DMS). We then retrained the rats for 15 s social interaction (FR1 schedule) for 16 d (8 d/sex) and recorded striatal dopamine during operant responding for a peer for 8 d (4 d/sex). Finally, we assessed economic demand by manipulating FR requirements for a peer (10 d/sex). In male, but not female rats, operant responding was higher for the opposite-sex peer. Female's estrous cycle fluctuations had no effect on operant social interaction. Striatal dopamine signals for operant social interaction were dependent on the peer's sex and striatal region (NAc core vs DMS). Results indicate that estrous cycle fluctuations did not influence operant social interaction and that NAc core and DMS dopamine activity reflect sex-dependent features of volitional social interaction.

The role of stimulus combinations in the repeated assessment of resurgence.

The current study examined the role of stimulus combinations on the repeated assessment of resurgence. Using a within-session resurgence procedure, rats were exposed to different conditions, each with distinct stimulus combinations (AAA, ABA, ABB, ABC and AAB). Two arrangements of stimulus changes were compared: Experiment 1 involved changes in stimulus combinations every five sessions, while Experiment 2 implemented changes every session. Resurgence was observed in both experiments; however, Experiment 2 demonstrated more consistent and repeated resurgence when stimulus combinations changed every session. Notably, the ABA, ABB and ABC conditions showed the highest percentage of sessions in which resurgence was observed. Lastly, the current study extends the application of the within-session resurgence procedure to rats and auditory stimuli, providing a reliable method for assessing resurgence in single subjects under different variable conditions.

Selective orexin 1 receptor antagonism does not affect effort-based responding for sucrose reward in rats.

In rodents, orexin neuropeptides regulate motivation and reward-seeking via orexin 1 receptor (OX1R) signaling in the mesolimbic dopaminergic system. This role is clearly established for rewards inherent to drugs of abuse but less so for natural rewards. Reported effects of the selective OX1R antagonist (SO1RA) SB-334867 on motivation for palatable food are ambiguous. In our experimental conditions neither SB-334867, nor two additional, structurally different SO1RAs, ACT-335827 and the clinical development candidate nivasorexant, affected effort-based responding for sucrose in rats. The positive control lisdexamfetamine, approved for psychiatric disorders associated with altered reward sensitivity such as binge eating disorder, increased effort-based responding.

Assessment of the inbred C57BL/6 and outbred CD1 mouse strains using a progressive ratio schedule during development.

Inbred strains have a genetic similarity of at least 98.6% compared to their outbred counterparts. Several studies have shown that inbred C57BL/6 mice and outbred ICR (CD1) mice differ in locomotion, cognitive flexibility, and aggression. However, their performance in operant paradigms is not well understood. A progressive ratio (PR) schedule of reinforcement is a method of quantitative estimation of the incentive state of an animal for a reward by increasing response requirements for reinforcer delivery, which is relevant to assess the breakpoint (amount of response effort an animal is willing to invest for a single unit of reward). This study tested male and female C57BL/6 and CD1 mice with an open field to analyze locomotion. Then, we used conditioning chambers with a PR3 schedule for ten consecutive days (P30-P40). PR performance was measured with the breakpoint, and the mathematical principles of reinforcement (MPR) were used to estimate motivation, impulsivity, and motor skills to manipulate the operandum. We found that CD1 mice showed higher locomotor activity than C57BL/6 independently of sex. CD1 mice had a higher breakpoint. However, male CD1 mice gradually increased breakpoint until the last session. In the MPR model, CD1 mice showed decreased fixed paused parameter (impulsivity) than C57BL/6, independent of sex. Our data suggest that the higher breakpoint in CD1 strain may partially be related to impulsivity. Therefore, the MPR model can help identify factors that affect performances, such as motivation, impulsivity, and motor skills during a PR in adolescent CD1 and C57BL/6 mice. These findings are essential to characterize the differences in the behavioral performance between C57BL/6 and CD1 strains and their potential as animal models.

Opening new vistas on obsessive-compulsive disorder with the observing response task.

Obsessive-compulsive disorder (OCD), a highly prevalent and debilitating disorder, is incompletely understood in terms of underpinning behavioural, psychological, and neural mechanisms. This is attributable to high symptomatic heterogeneity; cardinal features comprise obsessions and compulsions, including clinical subcategories. While obsessive and intrusive thoughts are arguably unique to humans, dysfunctional behaviours analogous to those seen in clinical OCD have been examined in nonhuman animals. Genetic, ethological, pharmacological, and neurobehavioural approaches all contribute to understanding the emergence and persistence of compulsive behaviour. One behaviour of particular interest is maladaptive checking, whereby human patients excessively perform checking rituals despite these serving no purpose. Dysfunctional and excessive checking is the most common symptom associated with OCD and can be readily operationalised in rodents. This review considers animal models of OCD, the neural circuitries associated with impairments in habit-based and goal-directed behaviour, and how these may link to the compulsions observed in OCD. We further review the Observing Response Task (ORT), an appetitive instrumental learning procedure that distinguishes between functional and dysfunctional checking, with translational application in humans and rodents. By shedding light on the psychological and neural bases of compulsive-like checking, the ORT has potential to offer translational insights into the underlying mechanisms of OCD, in addition to being a platform for testing psychological and neurochemical treatment approaches.